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Abstract. The likelihood filter for cluster detection in- 
troduced by Postman et al. is generalized by using stan- 
dard procedures and models originally developed in the 
theory of point processes. It is shown that the filter for- 
mulae of Postman et al. can be recovered in cases where 
background fields dominate the number counts. The per- 
formance of the generalized method is illustrated by using 
Monte Carlo simulations and by analyzing galaxy distri- 
butions extracted from the COSMOS galaxy catalogue. 
The generalized method has the advantage of being less 
biased at the expense of some higher computational effort. 
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1. Introducing Remarks 

The analogy between filtering and object detection in 
point distributions was recognized since more than 25 
years (e.g., Yashin 1970, Snyder 1972, Rubin 1972). Re- 
cent astronomical applications are given in, e.g., Dalton 
et al. (1994), Kawasaki et al. (1998), and Postman et 
al. (1996). In the latter project (Palomar Distant Clus- 
ter Survey, PDCS) clusters of galaxies are detected and 
their rcdshifts and richnesses are estimated by maximiz- 
ing the excess of the galaxy number counts as a function 
of angular coordinate and apparent magnitude with re- 
spect to the foreground and background galaxy distribu- 
tion. In the following both distributions are referred to as 
'background' distribution. The algorithm is based on the 
assumption that the statistics are dominated by the back- 
ground galaxy counts. In this 'high-background approxi- 
mation' the expected large numbers of galaxies ensure the 
validity of a Gaussian approximation of the correspond- 
ing likelihood function (central limit theorem) which is 
maximized to find the clusters and their basic parame- 
ters. It can be shown that this type of maximization corre- 
sponds to the minimization of the error-weighted squared 
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differences between observed and modeled data (method 
of least squares, e.g., Barlow 1996, p. 93). All counting er- 
rors are attributed to Poisson noise in the background. 
The method outlined above is optimized in finding dis- 
tant clusters where the approximations seem to be ap- 
propriate. In the following we describe a generalization 
of the PDCS filter where no a priori information about 
the contrast of the cluster distribution with respect to the 
background distribution is needed. We replace the Gaus- 
sian approximation of the likelihood function by an ex- 
act expression based on local Poisson distributions with 
position-dependent mean intensities guided by the pro- 
jected cluster number density profile and by the luminos- 
ity function. A similar approach based on binned data was 
developed independently by Kawasaki et al. (1998). Their 
finally used likelihood function, however, differs from the 
one derived in the present paper. 

The aim of the present investigation is to study a 
mathematically exact and general likelihood filter which 
can be further optimized and then applied to the detec- 
tion and characterization of isolated galaxy clusters lo- 
cated in comparatively uniform background galaxy fields. 
We plan to apply the method to galaxy fields centered 
around flux-limited samples of X-ray sources, e.g., to 
the ROSAT-ESO-Flux-Limited X-Ray (REFLEX) Clus- 
ter Catalogue (Bohringer et al., in preparation). The al- 
gorithm can be applied, however, to a much broader class 
of detection problems. The REFLEX clusters have mea- 
sured redshifts up to z = 0.30; a few exceptional cases 
have 0.30 < z < 0.50. The bulk of the data is located in 
the range 0.02 < z < 0.15. Within the REFLEX project, 
the filter can be used to compute statistical significances 
of cluster detections, and to estimate redshifts and rich- 
nesses of the optical counterparts. The redshift estimates 
could further support those cluster rcdshifts which are ob- 
tained spectroscopically with only small numbers of clus- 
ter galaxies. The richness estimates could give more infor- 
mation about selection effects introduced by the complex 
X-ray/optical survey process of the REFLEX project. 

The approach is based on the assumption that the 
spatial and the magnitude distributions of the galaxies 
located in the direction of clusters can be modeled by a 
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marked inhomogeneous Poisson point process. The result- 
ing likelihood function is given in a compressed analytic 
form using the concept of the likelihood ratio statistics 
(Sect.|). It is shown that the PDCS filter can be recov- 
ered in cases where background number counts dominate 
the statistic (Appendix B). Some practical notes on the 
application of the filter are given in Sec.|. In Sect. | 
the performance of the general method is illustrated by 
analyzing simulated data to measure possible statisti- 
cal biases of the derived numerical estimators (very of- 
ten maximum likelihood estimators are not unbiased; see, 
e.g., Ripley 1991, Sect. 4, and Barlow 1996, p. 84) and by 
analyzing subsamples of the COSMOS galaxy catalogue 
to test the methods under more realistic survey condi- 
tions. The results are summarized and the deviations from 
the idealized model assumptions are discussed in Sect. ||. 
All computations assume pressureless Friedmann-Lemai- 
tre world models with the Hubble constant, Ho, in units 
of h = iVtlOOkms^Mpc- 1 ), 

a negligible cosmological 
constant, and the deceleration parameter qo = 0.5. 



2. General Model 

The spatial distribution of the galaxies can be regarded 
as an inhomogeneous but in principle more or less random 
pattern of points, i.e., as a realization of a point process in 
mathematical terminology (Neyman & Scott 1952, Layzer 
1956, see also Neyman 1961 and references therein). The 
point process can be considered either as a random set 
of discrete points or as a random measure, counting the 
number of points in a given region (method of counts-in- 
cells). The points may be distributed in a three-dimen- 
sional volume, in a two-dimensional patch on the celestial 
sky, in a magnitude space, etc. 



2.1. Inhomogeneous Poisson Point Process 

All distributions with random locations of points and with 
local density parameters guided by a nonrandom variable 
fall into the category of inhomogeneous Poisson point pro- 
cesses. This is the case for the inhomogeneous galaxy dis- 
tribution seen in the direction of one galaxy cluster where 
the local more or less radial-symmetric variation of the 
projected galaxy number density profile plus the uniform 
background field are determined by the intensity \{x). The 
inhomogeneous Poisson point process is well-known in the 
theory of point processes (see, e.g., Cressie 1993, p. 650 
and references therein). If we replace the deterministic 
variation of the density parameter by a random variable 
we enter the field of doubly stochastic Poisson processes 
(Cox processes, see, e.g., Stoyan, Kendall, & Mecke 1995, 
p. 154) which are the general types of point processes to 
characterize large-scale distributions of galaxies (e.g., a 
Gaussian random field combined with local Poisson point 
processes). 



Let x\ , . . . , xn be one spatial realization of a point pro- 
cess. The N points, all with different coordinates, are dis- 
tributed within the total volume A. A useful quantity with 
a simple interpretation and a direct relation to powerful 
analytical tools provided by the theory of point processes 
is the local Janossy density, jN{%i, ■ ■ ■ ,%n\A): after mul- 
tiplication of j(-) with the product of the volume elements 
dxi ■ ■ ■ dxN it gives the probability that in the total vol- 
ume A there are exactly N points in the process, one point 
in each of the N distinct infinitesimal half-open regions 
[xi, Xi + dxi). The Janossy density may be regarded as the 
likelihood La of the realization, ignoring - as usual - the 
principle differences between probability density functions 
and sample functions: 



L A (xi, . . .,x N ) := j N (xi, . . . ,x N \A) . 



(1) 



Assume that the number of galaxies observed per unit 
solid angle depends only on the spatial coordinate x and 
gives the intensity X(x). More complicated models where 
also the magnitudes of the galaxies are taken into account 
are discussed b elow in the context of marked point pro- 
cesses (see Sec. 2/2). The intensity \(x) is considered as 
the x-dependent density parameter of an inhomogeneous 
Poisson process. 

The Janossy density and thus the likelihood function 
of the inhomogeneous Poisson point process can be de- 
rived directly by using the machinery of probability gen- 
erating functionals and Khinchin measures (Daley & Vere- 
Jones 1988, p. 498). The application of this formalism of- 
fers, however, the possibility to derive likelihood functions 
for more complex point processes, e.g., multiple stochas- 
tic point processes which might be interested for many 
cosmological applications. A guideline through the basic 
equations of the formalism is given in Appendix A. For 
inhomogeneous Poisson point processes this standard for- 
malism leads to the logarithmic likelihood (see also Snyder 
1975, Karr 1986, and Appendix A) 



In La 



\(x)dx 



N 

E 



In X(xi 



(2) 



Equation (|^) can also be found in a more heuristic way. 
As already mentioned above, the quantity La(x\, . . . ,xn) 
dx\ ■ ■ ■ dxN may be regarded as the probability of finding 
exactly one point in each of the infinitesimal volume ele- 
ments dxi, ... ,dxN, (case a), whereas outside of these re- 
gions no further points are present, (case b). These events 
are independent, due to the independence properties of 
the Poisson process. The probabilities for (a) are given by 
X(xi)dxi (for each individual point), and the probability 
for (b) is given by the avoidance or void probability func- 
tion, exp(— J X(x)dx). The integration extents over the 
volume A reduced by the sum of the infinitesimal volume 
elements, dxi, which can be neglected due to their small 
size so that, in total, the integration extents over the com- 
plete volume A (Stoyan & Stoyan 1992, p. 258). 
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2.2. Inhomogeneous Marked Poisson Point Process 

In order to generalize the model described above note 
that galaxy distributions can also be regarded as realiza- 
tions of marked spatial point processes, each consisting 
of locations of events in a bounded study region A and 
associated measurements (marks). Typical marks, often 
found in cosmological applications are apparent magni- 
tude, luminosity, mass, energy, color, morphological type, 
etc. In this case the realization of a marked point process 
is (xi, mi), . . . , (xn, tun). We assume the absence of any 
segregation effects, i.e., that the marks are independent - 
and-identically-distributed and are independent of the as- 
sociated marginal spatial point process. Define the point 
process by a counting measure on the product space AxB 
of two intervals with, for example, Xi G A C R 3 , and 
uij £ B C R 1 . The moment measures of a marked spa- 
tial point process are simple extensions of the moment 
measures of an ordinary spatial point process. The gener- 
alization of (J2J) to marked inhomogeneous Poisson point 
processes is thus straightforward and is given by 



In L A 



xLS 



A(x, m)dm dx 
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(3) 



Usually, A(x,m) can be factorized as A(x, to) = f(m\x)- 
\q(x) where f(m\x) is the conditional density of the 
marks, so that the log-likelihood can be maximized sepa- 
rately for the parameters characterizing / and Ao- 

2.3. Application to Cluster Detection 

In the case of finding clusters of galaxies using spatial 
and magnitude information A(x,to) cannot be factorized 
as can be seen from the model for the number of galaxies 
observed per unit area and per unit magnitude interval 
introduced by Postman et al. (1996), 



A(r, m) := b(m) + A <p(m — to*) P — 

r r. 



(4) 



Here, b(m) gives the differential magnitude number counts 
of the background galaxies, <j>(m — to*) is the luminosity 
function (e.g., in the Schechter prescription) of one cluster 
shifted along the apparent magnitude scale in accordance 
with the given redshift parameter and superposed onto 
the background distribution, P{r/r c ) is the cluster an- 
gular surface number density profile as a function of the 
projected (radial) distance r from the center of the cluster 
with the projected characteristic radius r c , and A is a di- 
mensionless parameter characterizing the intrinsic cluster 
richness. Equation (^) assumes a uniform, i.e., unclustered 
background galaxy distribution, and that all clusters have 
the same luminosity function and the same spatial num- 



ber density profile. For the model (^) with given b(m), and 
the normalization (if the integrals do not diverge) 



(5) 



Mm — to*) P — dm d r = 1 . 
1 A J b \r c J 

equation (^) can be written as 



lni 



b(m) dmd 2 r — A + 



N 



V" In I &(to*) + A0(TOj - m*)P 



•/v 



(6) 



It is convenient to normalize likelihood functions using the 
concept of likelihood ratio statistics. Further notes on the 
scope of adaptability of the likelihood ratio statistics, espe- 
cially when not all random variables may be independent, 
can be found in Sen & Singer (1993, p. 72). For the present 
situation where b(m) is assumed to be known the results 
do, however, not explicitly depend on the specific normal- 
ization. As a reference process in the denominator usually 
the homogeneous Poisson process is chosen. In order to get 
a likelihood ratio where its maximization explicitly maxi- 
mizes the contrast between cluster and background, an in- 
homogeneous Poisson process with an r-independent den- 
sity as given in ^ for A = (background model) seems to 
be more appropriate. Note that although the point distri- 
bution of this specific model is homogeneous in the spatial 
domain it is still inhomogeneous in the magnitude space. 
The corresponding log-likelihood is 



lnL = — I / b(m) dmd 2 r + \_] ln&(m,) 

J A JB 



(7) 



With this normalization the final log-likelihood ratio is 



JY 



, / L\ . ^ / A0(to,-to*)P(^; 
In — = -A + > In 1 + — ^ 



(8) 



The richness parameter A is obtained from the relation 
d\n(L/L>o)/dA = 0, resulting in the condition: 



N 

E 



0(to 4 -to*)p(^) 
ft(mi) + A0K -to*)p(^) 



= 1. 



(9) 



Equations (||) and (||) can be used for the detection of clus- 
ters in the following way: In the first step, values for to* 
and r c are selected so that, for given functional forms of 4> 
and P, equation (^|) gives the corresponding richness pa- 
rameter using standard numerical root finding algorithms 
(see, e.g., Brent 1973, Sect. 3 and 4). In the second step, 
this A value is inserted into (||) where ln(L/L ) serves 
as a significance measure for the detected cluster candi- 
date. The relations between the observed and the distant- 
independent values of the filter parameters are determined 
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by the redshift z and the chosen cosmological model. The 
final cluster redshift and richness values are thus fixed 
by the z value with the highest log-likelihood ratio. From 
A = it can be deduced that for a general galaxy field, 
ln(L/Lo) = is the mode of the distribution of the log- 
arithmic values of the likelihood ratios. The distribution 
is asymmetric with a long tail to larger ln(L/Lo)-values 
where the rich clusters are expected. Approximations of 
the equations (||) and @ are given in Appendix B. 

3. Practical Considerations 

For the application of the filter the intervals A and B in 
(|5j) must be specified. The following computations assume 
the half-open r- and m-ranges 



A = [0,r c 



D 



[TObright, TOf a i n t) . 



(10) 



In equations (|10|) r co is the halo or cutoff radius defined by 
P(r/r c ) — for r > r co , and P(r/r c ) > for r < r co . The 
bright and the faint-end limit of the apparent magnitudes 
of the sample galaxies are mbright and mf a i n t, respectively. 
For radial-symmetric cluster profiles, equations (|l0|), (Q), 
and (0) give a second estimator for the cluster richness, 



A = 2tt 



(11) 



"Ifaint 

r I X(r,m)dmdr 

' i J mbright 

Tlfaint 

2;: / r / b(m)dmdr = N - N hg = N ci 

J m bright 



With the specific choice (|10|), A is the difference between 
the total number of galaxies, N , seen in the direction of 
the cluster, and the number of background galaxies, A^g, 
expected for the cluster area determined by r co . Therefore, 
A is a statistical measure of the number of cluster galax- 
ies, N c i, detected in the magnitude range B above the 
background distribution. The estimator gives richnesses 
which depend on cluster redshift. Note the difference be- 
tween A defined as a parameter characterizing the intrin- 
sic cluster richness by equation (Q) without any constrains 
on the intervals A and B, and A defined as an estimator 
of the cluster richness through equation ( |l~l| ) under the 
constrains (|l0|). Equation ([ll]) not only shows that esti- 
mates of cluster richnesses obtained with (^) or ([ll]) are 
biased. The equation also offers a new way to circumvent 
the more complicated estimation of A by solving condition 
(^). Richnesses obtained with ( pd| ) are, however, not ob- 
tained with the maximum likelihood principle. Therefore, 
the combination of A with (||) does not necessarily yield 
consistent maximum likelihood values. 

A redshift-independent and thus unbiased estimate, 
Ao, of the intrinsic cluster richness may be obtained from 
the ratios 



A 

Ao 



m fain t 

mbright 



(m — m*)dm 



(m — m*)dm 



mbright 



(12) 



where A is given by (^), and the cutoff magnitude, m co , 
used as the upper limit of the numerical integration of the 
luminosity function, must be chosen in accordance with 
the cluster redshift. This magnitude may be defined by 



m co (z) := m*{z) + Am [mag 



(13) 



Here, m*(z) is determined by the cluster redshift and 
m co (z) by a predefined increment, Am magnitudes fainter 
than m*(z). The last equality in (|l2| ) uses the normaliza- 
tion (||) separately for the radial and for the magnitude 
profile: 



P | — 1 2 7T r dr = 1 



(to — to*) dm = 1 



mbright 



' mbright 



(m — m*)dm 



The equations ( jl2| ) can also be used to correct A for red- 
shift biases. Finally, it should be noted that TObright must 
be bright enough to include the brightest galaxy of the 
sample. 

4. Filter Performance 

The following computations illustrate the basic properties 
of the likelihood filter. The model is given by the equa- 
tions (§,(§), ©, (|l|), and (|l|). All numerical simula- 
tions and reductions are performed for the photographic 
bj passband. Observed data are taken from the COSMOS 
galaxy catalogue (e.g, Heydon-Dumbleton et al. 1989). 

For the magnitude filter, <j>, a Schechter-type luminos- 
ity function is used with the characteristic magnitude, 
M* = —20.1 + 5 log h [mag] , and the faint-end slope, 
a = -1.2 (Colless 1989, Lumsden et al. 1997). The COS- 
MOS magnitudes are corrected for the effect of the limited 
dynamic range within the microdensitomcter ('saturation 
effect') as in Lumsden et al. (1997, Sect. 2.2). The cosmic 
(K + E) corrections for the bj passband are parameter- 
ized as K(z) — 3.0z [mag] (Efstathiou, Ellis, & Peterson 
1988; Dalton et al. 1997). If the galaxy magnitudes have 
large random errors one has to replace cj>(m — to*) by the 
luminosity function convolved with the magnitude errors. 

The corrections for galactic extinction are applied to 
observed data using the correlation between visual extinc- 
tion and neutral Hydrogen column density, Nh [cm -2 ] j Ay 
= 1.79 x 10 21 (Predehl & Schmitt 1995). The visual ex- 
tinctions are transformed to the B band using the colour 
excess Eb-v = Ab — Ay and As — 4.3£ , s_y (de Vau- 
couleurs et al. 1991, p. 30). It is assumed that the ex- 
tinction in the B band closely resembles the effect in the 
bj passband. Hydrogen column densities are taken from 
Dickey & Lockman (1992) as provided by the EXSAS97 
release. 

For the radial filter, P, a King-like profile as given in 
Postman et al. (1996) is used, 
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Fig. 1. Abell cluster richness _Rab determined from simula- 
tions as a function of the intrinsic cluster richness A, i.e., the 
number of cluster galaxies (above background) with (as an ex- 
ample) absolute magnitudes M < — 11 + 51og(ft) [mag]. The 
la error bars are the standard deviations obtained from sim- 
ulations of clusters at different redshifts. The simulations give 
galaxy distributions as expected from the COSMOS galaxy 
catalogue. 



Fig. 2. Log-likelihood ratios for clusters (points) simulated 
with intrinsic richnesses 100 < A < 2000 in the redshift range 
0.04 < z < 0.24, marked on the right-end of each sequence. 
The short-dashed line shows the (formal) 3<r detection limit. 
The vertical error bar at zero richness indicates the 3<r range 
of the ln(L/I/o) values obtained for simulated fields without 
superposed clusters of galaxies. 



P(b) 



(1 + 6 2 ) >-(l + C 2 ) 3 : b<r/r co 
: b>r/r co ' 
with b = r/r c and C = r co /r c . The core radius and the 
cutoff radius have the respective values r c — 100ft. _1 kpc 
and r co = l/i" 1 Mpc. For the richness estimates using 
equation ( [l2| ) the magnitude increment Am = 3.0 mag in 
( plf ) was chosen. 

In the following paragraphs the detection of individ- 
ual clusters and the estimation of cluster richnesses are 



discussed (Sect. 4.1). For rich clusters the likelihood filter 
derived in Sections | and |^ gives unbiased estimates of the 
cluster redshift, as illustrated by the application of the 
filter to simulated data (Sect. p~2| ) and to observed data 
(Sect.O). 
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4-1- Cluster Richness and Cluster Detection from 
Simulations 

Although not essential for the application of the likelihood 
filter, knowledge of the relation between the traditional 
Abell cluster richness, Rab, and the not observable intrin- 
sic cluster richness, A, defined by equation (Q) is helpful in 
order to understand the efficiency of cluster detection and 
cluster characterization. Isolated clusters with known red- 
shifts, equatorial coordinates, and richnesses, superposed 
onto an homogeneous background galaxy distribution, are 



Fig. 3. Richness estimates A (declining sequence) and Ao (hor- 
izontal sequence) as a function of redshift z from simulated 
clusters of galaxies. Richnesses A are computed with equa- 
tion (^) ; richnesses Ao are computed with equation (jl2|) . Both 
richness estimates are normalized at z = 0.04. The error bars 
correspond to the la standard deviations. The errors are com- 
paratively small because of the high richness (A = 2000) used 
in the simulations. 
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Fig. 4. Richness estimates Ao obtained with the likelihood 
filter from simulated clusters with redshifts z < 0.24 as a func- 
tion of intrinsic richness A (defined by equation^). Error bars 
correspond to la standard deviations. 



600 



400 




200 - 



500 1000 1500 
A (M B ^-ll + 51og h) 



2000 



Fig. 5. As Fig.0 for clusters with redshifts z < 0.16. 



simulated with simple Monte Carlo methods in relativis- 
tic spacetime. The galaxies have apparent magnitudes in 
the range 0.0 < bj < 20.5 mag. The background number 
counts, b(m), are guided by polynomial fits to the differen- 
tial distributions collected by Metcalfe et al. (1993). The 
luminosity functions and the projected radial number den- 
sity profiles of the simulated clusters are the same as those 
used in the likelihood filter. The results obtained from nu- 
merical simulations are shown in Fig. |} For each intrinsic 



richness A, the mean Abell richnesses (crosses) and the 
la error bars are obtained from clusters simulated at dif- 
ferent redshifts. The analysis concentrates on the Abell 
richness classes —1,0,1, and 2 because most of the clusters 
we want to study have these comparatively low richnesses. 
The nonlincarity between the Rab and the A richnesses as 
seen in Fig. [I] is introduced by the richness-dependency of 
ni3, that is, the apparent magnitude of the 3rd- brightest 
cluster galaxy (Scott effect). The large scatter of the in- 
dividual richness estimates is mainly caused by the large 
scatter of the values. 

Figure H shows the efficiency of the likelihood filter for 
cluster detection as a function of the intrinsic cluster rich- 
ness A and for different cluster redshifts, given at the right- 
end of each sequence in the figure. The 3a range of the 
\tx{L/Lq) values obtained for pure background fields with- 
out superposed galaxy clusters is indicated by the vertical 
error bar at A = 0. As expected, the log-likelihood values 
increase with increasing cluster richness and decreasing 
cluster redshift. The simulations suggest that if the clus- 
ters have the same luminosity function and radial num- 
ber density profile as used in the likelihood filter then the 
method would detect all clusters in the COSMOS galaxy 
catalogue with redshifts z < 0.24 and richness A > 500 
or i?AB > 40 (Abell richness class 0) on significance lev- 
els > 3a. For larger redshifts, however, only clusters with 
larger richnesses can be detected. 

The richness estimates A obtained with equation (JsJ) 
and the richness estimates Ao obtained with ( |l2"| ) are com- 
pared in Fig. [| As shown in Sect.|^, A estimates are al- 
ways redshift-dependent when they are computed within 
fixed magnitude intervals. They give the number of cluster 
galaxies above the background field: this number decreases 
with increasing z. Contrary to this, Ao estimates are cor- 
rected for the effect of the redshift-dependent magnitude 
range of the cluster galaxies. These estimates are almost 
constant over the given z interval. 

The accuracies of richness estimates are always lim- 
ited by principle problems concerning the determination 
of the cluster memberships and by the small numbers of 
detectable galaxies in distant clusters. These problems are 
illustrated in the Figs.^ and ||. The richness estimates A 
are computed for clusters at different redshifts and intrin- 
sic cluster richnesses A. The number of simulated clusters 
is constant per unit redshift interval. As expected, the 
random errors of the Ao estimates increase significantly 
for poor clusters at high z. Moreover, cluster richnesses 
are systematically overestimated for samples with high z. 
Only for cluster samples with small z and high A we can 
expect unbiased estimates which are directly linked to the 
intrinsic cluster richnesses. 



4-2. Tests for Redshift Biases from Simulations 

The following tests are devoted to studies concerning pos- 
sible systematic redshift errors which might be inherent 
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Fig. 6. Examples of log-likelihood curves from simulated data. 
The intrinsic richness of the clusters is A = 10 4 . The redshifts 
of the clusters are given on top of each panel. Formal standard 
deviations of the redshift estimates can be obtained from the 
points where the log-likelihood curve has fallen by 0.5 (lcr), 2.0 
(2a), 4.5 (3a), . . . from its maximum. 

for the derived method. In order to detect z biases with- 
out the disturbing effects of the discreteness noise, the 
analysis begins with the simulations of clusters with high 
intrinsic richnesses, A = 10 4 . We regard these simulations 
as some kind of numerical integration of the relevant equa- 
tions. At the end of this paragraph, the results from the 
simulations of clusters with low richnesses are discussed. 

Figures^] show the likelihood values obtained for four 
rich clusters simulated at the redshifts z = 0.02. 0.14, 0.20, 
and 0.26. As already shown in the preceding sections, the 
magnitude and redshift ranges of these simulations give 
galaxy distributions as selected from the COSMOS galaxy 
catalogue. 

For all redshifts the log-likelihood curves in Fig. || peak 
close to the true cluster redshifts (indicated on top of each 
panel). For intermediate z values, the comparatively high 
symmetry of the curves around the correct cluster red- 
shift indicates the absence of large systematic z errors. For 
small and for large redshifts the curves get more asymmet- 
ric. This might be taken as a hint to possible increasing 
systematic z errors when the method is applied to com- 
paratively nearby or distant clusters. 

The asymmetries, however, do not significantly affect 
the positions of the maxima of the log-likelihood curves 
and thus of the derived redshifts. For example, the dif- 
ferences, Az, between the true redshifts and the likeli- 
hood redshifts are \Az\ = 0.0005 for the extreme redshifts 
z = 0.02 and z = 0.26, and do not exceed \Az\ = 0.0025 
for all cases studied so far. For very rich clusters, the Az 



Fig. 7. Comparison of the redshift estimates obtained with 
the likelihood filter, ^Likelihood, with the true cluster redshifts, 
Zsim, for different intrinsic cluster richnesses (A value given 
in the upper left of each panel). The data are obtained from 
simulations. The richness A = 8000 corresponds to a very rich 
cluster whereas A = 500 characterizes a cluster which is near 
the detection limit of the likelihood filter expected for the sim- 
ulated COSMOS data when the cluster is located at z ~ 0.24. 

values do not show any z-dependent trends, and are of the 
order of the expected numerical accuracies. 

We thus conclude that numerical studies under ideal 
conditions (high richness, and background distribution as- 
sumed to be known) give no indications for possible z- 
dependent biases of the locations of the maxima of the 
log- likelihood curves. 

The situation changes for poor clusters and when the 
background number counts must be estimated from the 
sample itself. The comparison of the likelihood redshifts 
with the true cluster redshifts is shown for different in- 
trinsic richnesses in Fig.|^. For high richnesses no system- 
atic redshift errors are detected, supporting the results 
of the analyses described above. For poor clusters with, 
e.g., A = 500 (i.e., clusters close to the detection limit at 
z = 0.24) a systematic overestimation of z at high red- 
shifts is found. This is mainly caused by the small num- 
ber of cluster galaxies detected for poor clusters at high z 
leading to a systematic undersampling of the bright end 
of the cluster luminosity function and of the outskirts of 
the projected cluster profile. 

4-3. Redshifts from Observed Data 

Before we describe the results obtained with the likelihood 
filter on observed data a few comments regarding some 
practical limitations are given. The derivation of the like- 
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Fig. 8. Redshifts of observed clusters of galaxies, 
^likelihood, obtained with the likelihood filter using 
subsamples of the COSMOS galaxy catalogue, and spectro- 
scopic cluster redshifts, Zesokp, obtained with eight to 20 
galaxies per cluster as published in the REFLEX catalogue of 
Bohringer et al. (in preparation). 

lihood filter rests on the assumption of an homogeneous 
and isotropic spatial distribution of the background galax- 
ies. In mathematical terminology this involves the validity 
of first-order motion invariance, i.e., the invariance of the 
mean intensity of galaxies in the study area outside the 
cluster region under spatial translations and rotations. In 
many practical situations the observed data do not fulfill 
these requirements. 

On small scales, the galaxy field is strongly clustered so 
that comparatively large solid angles are needed to reduce 
biases in the background determination. On large scales, 
extreme care must be taken to achieve homogeneous sam- 
pling of the galaxy field. The latter aspect is important for 
the analysis of galaxy catalogues obtained from mosaics of 
digitized Schmidt plates where the effects of vignetting, 
desensitization, and small errors in magnitude zero-points 
may introduce artifical fluctuations in the observed galaxy 
field, especially when galaxy distributions obtained from 
different plates are combined. For the application of the 
likelihood filter to observed data it is thus necessary to ex- 
tract galaxy fields on areas which are not too small to be 
affected by galaxy number density fluctuations and which 
are not too large to be affected by artifical plate-to-plate 
variations. 

Tuckcy's median polishing offers a robust way to detect 
large-scale deviations from first-order motion invariance in 
point distributions. The algorithm successively subtracts 
medians out of rows of gridded data (number counts in 
quadratic cells), then columns, then rows, then columns, 
and so on, accumulating them in 'row', 'column', and 'all' 



registers, and leaves behind the table of residuals. The 
convergence of this procedure is discussed in Kemperman 
(1984). An illustration of the algorithm is given in Ap- 
pendix C. 

We use this algorithm after transforming the equato- 
rial object coordinates into Hammer- Aitoff equal-area co- 
ordinates to flag regions by searching for cells in the row 
and in the column registers which significantly deviate 
(> 5cr formal background noise) from the mean galaxy 
background distribution and exclude the corresponding 
plates from further analyses. The results obtained with 
median polishing sometimes strongly depend on the spe- 
cific orientation of the row and column axes. Therefore, 
we apply the algorithm using three different orientations. 
For each orientation, median polishing usually converges 
after three to five iterations. The iteration stops when the 
relative changes between two successive iteration steps are 
less than one percent. 

In Fig. H the redshift estimates obtained with the like- 
lihood filter on subsamples of the COSMOS galaxy cat- 
alogue are compared to cluster redshifts obtained from 
spectroscopic observations. The central cluster coordi- 
nates and the cluster redshifts are taken form the RE- 
FLEX catalogue (Bohringer et al., in preparation). The 
test sample includes 43 REFLEX clusters observed among 
others within the ESO Key Programme (e.g., Guzzo et 
al. 1995). These clusters have redshifts determined with 
eight to 20 cluster galaxies and are thus well-suited as 
redshift references. In addition, the presence of compar- 
atively strong X-ray signals from these sources and the 
absence of active galaxies further support the existence 
of real clusters in these areas. The test sample is not a 
representative subsample of the complete REFLEX cata- 
logue. It is biased to larger redshifts because the complete 
REFLEX sample includes also a large number of nearby 
clusters. 

All COSMOS galaxies with bj < 20.5 mag (after cor- 
rection for saturation effects and galactic extinction) lo- 
cated within spherical regions of about two degrees radius 
around the X-ray cluster positions are selected. After me- 
dian polishing and careful determination of the bright and 
of the faint magnitude limits of the galaxies in each study 
region, typically 10 000 galaxies (background + cluster) 
remain per cluster target. 

For 28 clusters from the list of 43 test clusters the fil- 
ter detects a maximum in the likelihood functions within 
the redshift range 0.0 < z < 0.35. The parameters of 
the likelihood filter are the same as used in the simula- 
tions. The corresponding ^likelihood values are shown 
in Fig.||. For the remaining 15 test clusters no redshifts 
could be determined, either because of their weak statisti- 
cal significance ln(L/L ) < 4.5 (two cases) or because no 
local maximum was detected (13 cases). These redshift 
incompletenesses increase with increasing redshift from 
20 to 25 percent for z < 0.2 to 53 percent for z > 0.2. 
However, the redshift distribution of the test clusters was 
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Table 1. Effects of false M* and r co filter parameters 
on redshift and richness estimates. M* values are given 
in units of 5 \og(h) mag, t co values axe given in units of 
Mpc/h (see text). 



M*\ r co 


0.5 




1.0 




2.0 




-18.1 


0.050 


1033 


0.075 


2080 


0.090 


3029 


-20.1 


0.064 


433 


0.109 




986 


0.148 


1754 


-22.1 


0.065 


206 


0.131 


425 


0.240 


1090 



Table 2. Effects of false M* and r c filter parameters on 
redshift and richness estimates. M* values are given in 
units of 5 log(h) mag, r c values are given in units of Mpc//i 
(see text). 



M*| r c 


0.05 




0.1 




0.5 




-18.1 


0.070 


1835 


0.075 


2080 


0.085 


2638 


-20.1 


0.106 


921 


0.109 


986 


0.120 


1121 


-22.1 


0.128 


420 


0.131 


425 


0.140 


457 



Table 3. Effects of substructure on the richness and red- 
shift estimates as a function of the separation Ar and 
for different richness ratios A1/A2 of the two components. 
The application of the general likelihood filter gives the 
normalized richness (numerator) and the redshift (denom- 
inator) for the combined system. 



^ I £l 1.000 0.500 0.250 0.125 0.025 0.000 



■J." r .'> 
0.104 
2.049 
0.104 
2.043 
0.101 
2.023 
0.102 
1.759 
0.095 
1.511 
0.104 



1.529 
0.105 
1.525 
0.105 
1.517 
0.104 
1.505 
0.105 
1.464 
0.103 
1.254 
0.102 



1.2(i5 
0.105 
1.409 
0.105 
1.255 
0.105 
1.246 
0.104 
1.229 
0.101 
1.188 
0.096 



1.130 
0.106 
1.259 
0.106 
1.257 
0.106 
1.122 
0.105 
1.110 
0.104 
1.086 
0.101 



1.026 
0.106 
1.025 
0.106 
1.025 
0.106 
1.024 
0.105 
1.022 
0.105 
1.016 
0.106 



1.000 
0.106 
1.000 
0.106 
1.000 
0.106 
1.000 
0.106 
1.000 
0.106 
1.000 
0.106 



constructed to be almost redshift-independent so that the 
given incompleteness values must be weighted with the a 
priori probabilities for the presence of clusters of galaxies 
within different redshift intervals (multiplication formula 
for probabilities). Therefore, the actual incompletenesses 
are expected to be much smaller compared to the values 
given above. The standard deviation of the redshift differ- 
ences (^likelihood - Zesokp) is (T z = 0.023 ± 0.004. 



5. Concluding Remarks 

Both the PDCS filter and the generalized filter derived in 
the present paper give systematic redshift errors for clus- 
ters at high z. The generalized filter overestimated red- 
shifts at high z; the errors tend to zero for richer systems. 
The PDCS underestimates redshifts at high z; the errors 
tend to zero for richer systems (see Fig. 14 in Postman 
et al.). Whereas the errors of the generalized method can 
be explained by the incomplete sampling of the filter pro- 
files for poor clusters at high z, the errors of the PDCS 
filter can not be judged in an easy way. Reasons for the 
descrepancies between the two methods are still not clear 
because of the complexity of the PDCS approximations. 
Nevertheless, it seems that if one uses the exact mathe- 
matical equations, as does the generalized method, correc- 
tions like the 'cluster signal correction' of the PDCS filter 
are not necessary. 

It it shown that the richness estimates obtained with 
the generalized filter are redshift-dependent and must be 
corrected (e.g., with equation |l2|). It is not clear why sim- 
ilar corrections are not necessary for the PDCS filter. 

Similar conclusions about the PDCS filter are drawn 
by Kawasaki et al. (1998). In contrast to the generalized 
method, the Kawasaki filter uses binned data and does not 
maximize the contrast between field and cluster galaxy 
counts. A detailed comparison of the generalized method 
and the approximations introduced by Kawasaki et al. is 
in preparation. 

The final question concerns the validity of the chosen 
assumptions used to derive the likelihood filters. Prob- 
lems caused by deviations of the spatial distribution of 
the background galaxies from the assu med uniform distri- 
bution were already discussed in Sect. |4.3| . 

Further systematic errors in the derived cluster red- 
shifts and richnesses arise when the luminosity functions 
and the number density profiles of the likelihood filter do 
not match with the corresponding functions of the ob- 
served clusters. As an example, Tabs.[j] and || illustrate 
the effects of false filter parameter values M*, r co , and r c 
on the estimates of z and Ao. The results are obtained 
from simulated data. The 'correct' cluster parameters, de- 
rived for the case when </> and P of the filter exactly match 
with the corresponding cluster functions, are shown as ref- 
erence in the tables within the small boxes. 

The ranges of the filter parameters are clearly far be- 
yond the variations expected for realistic cases. Neverthe- 
less, the results given in Tabs.|l| and |^ illustrate the stabi- 
lizing effect of using both spatial and magnitude informa- 
tion to derive the cluster z and Ao- For some parameter 
combinations, however, the systematic errors are ampli- 
fied. 

It is also seen that false values of the cutoff radius, r co , 
give larger systematic z and Ao errors compared to false 
values of the core radius, r c . One might expect that the 
stronger effect of r c on truncated cluster profiles would 
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also translate into higher sensitivities of z and Ao on this 
filter parameter. The conclusion, however, neglects the ef- 
fects of errors in the background number counts. The num- 
ber of the background galaxies expected for the cluster 
area scales with r^ so that small errors in r co have a large 
effect. 

In many clusters of galaxies the presence of substruc- 
tures may lead to significant systematic errors in the es- 
timated cluster richnesses and redshifts. In order to sim- 
ulate unrelaxed clusters in a stage of formation by merg- 
ing of subunits we approximate such configurations by su- 
perposing two cluster models at close separation which 
seems to describe observed mergers quite well (e.g., Bricl 
et al. 1991, Bohringer et al. 1996). The two components 
are supposed to have the same redshift and therefore the 
same number density profiles and luminosity functions, 
but varying richness ratios are chosen. Tab.[| summaries 
the results obtained for different distances Ar (in units of 
r c ) between the centers of the two components, and for 
different ratios, Ai/A 2 , of the richnesses. The table gives 
for the combined system the estimated richnesses (numer- 
ator) normalized to the richness of the main component, 
and the estimated redshifts (denominator). As expected, 
the maximum likelihood filter gives for small Ar the ap- 
proximate sum of the two richnesses, converging for large 
Ar to the richness of the main component. Fortunately, 
the redshifts are almost uneffected by this type of sub- 
structure. 

The effects of different faint-end slopes of the lumi- 
nosity functions (not shown here) are significantly smaller 
compared to the effects mentioned above and can in gen- 
eral be neglected for the distant clusters. 

To summarize, the likelihood filter described here, is 
shown to be a useful method to detect and to charac- 
terize distant clusters of galaxies. Clusters with compar- 
atively low richnesses can be detected out to fairly large 
distances. Reliable redshift and richness estimates, can, 
however, only be obtained for the more nearby, richer, 
and isolated systems. 
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Appendix A: Derivation of Likelihood Functions 
for Point Processes 

In the first step, the probability generating functional Ga 
of a point processed, defined by the expectation 

G A [h] := E( J] Hxi)) . (A.l) 
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must be specified (e.g., Stoyan, Kendall, & Meek 1995, 
p. 116). Here, h(x) is a [0, 1]- valued (test) function. In 
(A.l) the point process on A is represented by N, and 
the points are located at the spatial positions {xi\. 

In the second step, the logarithmic generating func- 



tional obtained with (A.l) is compared with the expan- 
sion 

oo „ 

lnG A [h] = -Ko(A) + V (n!)" 1 / 

Jam 



h(xi) ■ ■ ■ h(x n ) K n (dxi ■ ■ ■ dx n \A){k.2) 
to get the Khinchin measures, K n . Som e re marks con- 



cerning the derivations of both equation (A.2) and ( A. 3 , 
see below) can be found in Daley & Vere- Jones (1988, 
Sect. 5.5, and p. 230). 

In the last step, the K n measures are inserted into the 
exansion of the local Janossy densities, 

j N (xi, . . . ,x N \A) = 

N r 

e ~ K °J2 E II k \s t w\(xi,i,---,Xi,\ Si <T)\\A) , (A.3) 

r=l TeV rN i=l 

to get in combination with equation (|l]) the likelihood 
function of the point process. The second sum on the right- 



hand side of (A.3) is taken over all r-partitions P r N where 
|5j(r)| gives the number of elements in each partition set. 

Example: Inhomogeneo us P oisson point process. 
For this process, equation (|A. l| ) gives (Cressi 1993, p. 650) 



\nG A [h] = - [ (1 - h(x))X(x 

J A 



dx . 



(A.4) 



With equation QA.2] ) we obtain the only nonzero Khinchin 
measures, Kq = j A X(x)dx and k\{x\A) = X(x), resulting 
to the Janossy density 



N 



j N {x u ...,x N \A) = e- Ko Y[ ki(xi\A), 



(A.5) 



which directly leads to the likelihood function of the in- 
homogeneous Poisson point process (eq. || in Sect. |2.l| ). 



from which a closed analytic form for the cluster richness 
is obtained 



N 4>(mj—m") D/ n 



A 



i=l 



b( mi ) 



m) i 



L-ii=\ b 2 ( TO .) r y r j 



(B.2) 



In the same limit M) reduces to 







(i) 







A? 



(mi - to*) ( rj_ 



b(m l 



(B.3) 



Inserting (p3.2j) into (B.3) gives an analog equation to 18|) 



in the high-background approximation 

( X^ N H m i- m *) p ( rA -I V 
L \ \l^i=l b(mi) — ^ \—J ~ L ) 



hi 



2 (mj-m*) p2 f Ti \ 



(B.4) 



It is seen that equations (BT) and flB.2| ) correspond to 
equations (15) and (14) in Postman et al. (1996), respec- 
tively, if 

" <f> 2 (mi-m*)P 2 (a) 



A J B 



and 



N 

E 



6 2 K) 

<P 2 (m-m*)P 2 (^) 
b 2 (m) 

<\> 2 {m-m*)P 2 (£) 
b(m) 



A(r, to) dm d 2 r 
dm d 2 r , 



(B.5) 
(B.6) 



b(mi) 



A JB 



b(m) 



A(r, to) dm d 2 r , 



(B.7) 



and the normalization (|^) holds. 



Appendix B: High-Background Approximation 

Equations (|sj) and @ are mathematically exact and are 
based on no special mathematical approximation. In this 
section it will be shown that the equations of the digital 
filter derived by Postman et al. (1996) - not the finally 
equations used - can be obtained if 

b(m) » K(j)(m-m*)P ^— ^ . (B.l) 

In this limit the left side of the condition (|9|) may be ap- 
proximated by 

y, (f>(rm - to*) p fr£\ 4> 2 (mj -m*) p2 ( rA 



Appendix C: Median Polish Algorithm 

In the present investigation median polishing is used to 
identify those photographic plates where the surface num- 
ber density of galaxies deviates significantly from the cen- 
tral plate containing the major part of the galaxy cluster, 
so that the galaxies on those 'pathological' plates can be 
rejected from further analyses. The algorithm is, however, 
more powerful and thus interesting enough to be described 
in more detail. The method can, for example, be used for 
samples of sufficiently large sizes as a definition procedure 
of stationary point distributions. 

Let Nki, k — 1, . . . , p; I — 1, . . . , q be the number of 
points counted in the cell with the indices k and I. The 
k and the I index may number the count cells along the 
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Fig. C.l. Angular distribution of the COSMOS galaxies with 
b.j < 20.5 mag centered around the ROSAT X-ray source 
RX063720.4-482844. The data are collected from four different 
Schmidt plates. Multiple detections of galaxies in the overlaps 
of adjacing plates are rejected. The redshift of the central clus- 
ter of galaxies is z — 0.2019 measured in the course of an ESO 
Key Programme (Guzzo et al. 1995). 

Right Ascension and along the Declination axis, respec- 
tively. We follow the terminology of Cressi (1993, p. 186) 
and define for i = 1, 3, 5, . . . , 



N, 



N, 



- med^*" 1 ' | i = !,...,<?}, 



k = 1, . . . ,p + 1; 1 = 1, 



(C.l) 



N, 



W 



N, 



(i-i) 



k, g +i - -M+i Tmed K '\i = i,---,q}, 

k = l,...,p+l, (C.2) 



and define for i = 2, 4, 6, 



N, 



(i) 



m - Nt 1] -med{N^\k = l, 
k = l,...,p: l = l,...,q+l, 



(C.3) 



: = N p+ii + ^i N kl "'\k = l,...,p}, 
I = l,...,q + l. (C.4) 

Here, med{xi, . . . , x n } is the median of {x%, . . . , x n }. The 
algorithm starts with 



r(i-i) 



r(i-l) 



N, 



(0) 

kl 



k = l,...,p; 1 = 1, 



(C.5) 



: elsewhere. 
Usually the method converges after three to five iterations. 
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Fig. C.2. As Fig. C.l after median polishing. The weak indi- 
cation of a smaller surface number density in the north com- 
pared to the south might be attributed to a small error in the 
magnitude zero-point. Median polishing was adjusted to ac- 
cept such large-scale deviation from stationarity which are not 
uncommon in the COSMOS data base. Note that the excised 



region is smaller than the dark region in Fig. C.l due to the 
plate overlap. 



After median polishing the original p x q data matrix 
is replaced by a p x q residual matrix, Am, and by the 
P + q + 1 extra elements containing the deviations along 
the rows, {r^}, the deviations along the columns, {q}, 
and the average background level, {a}. These quantities 
are related by 



N kl = a + r k + a + A ki , 
k = l,...,p; 1 = 1,. ..,q. 



(C.6) 



Equation ( JC.q ) shows that a describes the overall median 
surface number density of galaxies in the area covered 
by the spatial indices k and I. Correspondingly, the pro- 
jected median surface number density profile in excess to 
the global median a along, e.g., the Right Ascension and 
the Declination axes, is thus given by the r k and by the 
ci vector, respectively. The residual matrix A^z gives the 
remaining deviations from (first-order) stationarity. The 
three effects are unbiased estimates if the coordinates of 
the individual galaxies are transformed into equal-area 
Hammer Aitoff coordinates. 

Figures C.l and |C.2| illustrate the performance of the 



filter. In this example th e CO SMOS galaxy data of four 
adjacing Schmidt plates (C.l) are combined. Median pol- 
ishing detects the artifical inhomogeneity at R.A. > 100.5 
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degrees, rejects the corresponding data from further anal- 
yses, and starts again with a new combination of the COS- 



MOS data from the remaining Schmidt plates (C.2). The 
algorithm can be regarded as a standard procedure in spa- 
tial data analyses and is thus well studied both mathemat- 
ically and in many practical applications (see the examples 
and references collected by Cressi 1993). 



